Method and system for optimizing comfort of an occupant

ABSTRACT

Method and apparatus for optimizing the comfort of the occupant in accordance with the invention in which the direction of a bothersome stimulus, such as an unwanted sound, external or internal of the vehicle, and an external light source, is determined, the location of the sensory organ of the occupant receptive to the stimulus, e.g., the ears of the occupant if the stimulus is an unwanted sound and the eyes of the occupant if the stimulus is an external light source, is determined and, responsive to the determined direction of the stimulus and the determined location of the sensory organ of the occupant receptive thereto, a countereffect to the stimulus is activated in order to diminish the effect of the stimulus on the sensory organ of the occupant. If the stimulus is the external light source, a light filter between the external light source and the eyes of the occupant may be activated. If the stimulus is a source of unwanted sound, the location of the ears of the occupant is determined and sound waves are generated and transmitted to cancel the unwanted sound at the ears of the occupant. The stimulus may be both sound and a light source.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 08/905,876 filed Aug. 4, 1997, now U.S. Pat. No. 5,848,802,which in turn is a continuation application of U.S. patent applicationSer. No. 08/505,036 filed Jul. 21, 1995, now U.S. Pat. No. 5,653,462,which in turn is a continuation-in-part of U.S. patent application Ser.No. 08/640,068 filed Apr. 30, 1996, now U.S. Pat. No. 5,882,782, whichin turn is a continuation-in-part of U.S. patent application Ser. No.08/239,978 filed May 9, 1994, now abandoned, a continuation-in-part ofSer. No. 08/040,978 filed Mar. 31, 1993, now abandoned, which is acontinuation-in-part of U.S. Ser. No. 07/878,571 filed May. 5, 1992, nowabandoned.

BACKGROUND OF THE INVENTION

In 1984, the National Highway Traffic Safety Administration (NHTSA) ofthe U.S. Department of Transportation issued a requirement for frontalcrash protection of automobile occupants. This regulation mandated"passive occupant restraints" for all passenger cars by 1992. A morerecent regulation requires both driver and passenger side airbags on allpassenger cars and light trucks by 1998. In addition, the demand forairbags is accelerating in both Europe and Japan and it is expectedthat, within a few years, all vehicles produced in these areas (36million vehicles) and eventually worldwide (50 million vehicles) will beequipped with airbags as standard equipment.

Whereas thousands of lives have been saved by airbags, significantimprovements can be made. As discussed in detail in U.S. Pat. No.5,653,462 cross-referenced above, and included herein by reference, fora variety of reasons, vehicle occupants can be or get too close to theairbag before it deploys and can be seriously injured or killed upondeployment of the airbag.

Also, a child in a rear facing child seat, which is placed on the rightfront passenger seat, is in danger of being seriously injured if thepassenger airbag deploys. This has now become an industry-wide concernand the US automobile industry is urgently searching for an easy,economical solution, which will prevent the deployment of the passengerside airbag if a rear facing child seat is present. An improvement onthe invention disclosed in the above-referenced patent, as will bedisclosed in greater detail below, includes more sophisticated means toidentify objects within the passenger compartment and will solve thisproblem.

Initially, these systems will solve the out-of-position occupant and therear facing child seat problems related to current airbag systems andprevent unneeded deployments when a seat is unoccupied. Airbags are nowunder development to protect rear seat occupants in vehicle crashes. Asystem will therefore be needed to detect the presence of occupants,position, i.e., determine if they are out-of-position, and type, e.g. toidentify the presence of a rear facing child seat in the rear seat.Future automobiles can be expected to have eight or more airbags asprotection is sought for rear seat occupants and from side impacts. Inaddition to eliminating the disturbance of unnecessary airbagdeployments, the cost of replacing these airbags will be excessive ifthey all deploy in an accident. The improvements described belowminimize this cost by not deploying an airbag for a seat, which is notoccupied by a human being. An occupying item of a seat may be a livingoccupant such as a human being or dog, another living organism such as aplant, or an inanimate object such as a box or bag of groceries.

A device to monitor the vehicle interior and identify its contents isneeded to solve these and many other problems. For example, once aVehicle Interior Identification and Monitoring System (VIMS) foridentifying and monitoring the contents of a vehicle is in place, manyother products become possible including the following:

Inflators and control systems now exist which will adjust the amount ofgas flowing into or out of the airbag to account for the size andposition of the occupant and for the severity of the accident. Thevehicle identification and monitoring system of this invention willcontrol such systems based on the presence and position of vehicleoccupants or of a rear facing child seat.

Side impact airbag systems began appearing on 1995 vehicles. The dangerof deployment induced injuries will exist for side impact airbags asthey now do for frontal impact airbags. A child with his head againstthe airbag is such an example. The system of this invention willminimize such injuries.

Future vehicles may be provided with a standard cellular phone as wellas the Global Positioning System (GPS), an automobile navigation orlocation system, which is now available on at least one vehicle model.In the event of an accident, the phone may automatically call 911 foremergency assistance and report the exact position of the vehicle. Ifthe vehicle also has a system as described below for monitoring eachseat location, the number and perhaps the condition of the occupantscould also be reported. In that way, the emergency service (EMS) wouldknow what equipment and how many ambulances to send to the accidentsite.

Vehicle entertainment system engineers have stated that the quality ofthe sound in the vehicle could be improved if the number, size andlocation of occupants and other objects were known. This information canbe provided by the vehicle interior identification and monitoring systemof this invention.

Similarly to the entertainment system, the heating, ventilation and airconditioning system (HVAC) could be improved if the number, attributesand location of vehicle occupants were known. This can be used toprovide a climate control system tailored to each occupant, for example,or the system can be turned off for certain seat locations if there areno occupants present at those locations.

In some cases, the position of a particular part of the occupant is ofinterest such as: (a) his hand or arm and whether it is in the path of aclosing window so that the motion of the window needs to be stopped; (b)the position of the shoulder so that the seat belt anchorage point canbe adjusted for the best protection of the occupant; or, (c) theposition of the rear of the occupants head so that the headrest can beadjusted to minimize whiplash injuries in rear impacts.

The above applications illustrate the wide range of opportunities, whichbecome available if the identity and location of various objects andoccupants, and some of their parts, within the vehicle were known. Oncethe system is operational, it would be logical for the system to alsoincorporate the airbag electronic sensor and diagnostics system (SDM)since it needs to interface with SDM anyway and since they could sharecomputer capabilities which will result in a significant cost saving tothe auto manufacturer. For the same reasons, it would be logical forVIMS to include the side impact sensor and diagnostic system. As theVIMS improves to where such things as the exact location of theoccupants ears and eyes can be determined, even more significantimprovements to the entertainment system become possible through the useof noise canceling sound, and the rear view mirror can be automaticallyadjusted for the driver's eye location. Another example involves themonitoring of the driver's behavior over time which can be used to warna driver if he or she is falling asleep, or to stop the vehicle if thedriver loses the capacity to control it.

Using an advanced VIMS, as explained below, the position of the driver'seyes can be accurately determined and portions of the windshield can beselectively darkened to eliminate the glare from the sun or oncomingvehicle headlights. This system uses electro-chromic glass, a liquidcrystal coating on the glass, or other appropriate technology, anddetectors to detect the direction of the offending light source. Inaddition to eliminating the glare, the sun visor can now also beeliminated.

The present invention adds more sophisticated pattern recognitioncapabilities such as fuzzy logic systems, neural network systems orother pattern recognition computer based algorithms to the occupantposition measurement system disclosed in the above referenced copendingpatent application and greatly extends the areas of application of thistechnology. An example of such a pattern recognition system using neuralnetworks using sonar is discussed in two papers by Gorman, R. P. andSejnowski, T. J. "Analysis of Hidden Units in a Layered Network Trainedto Classify Sonar Targets", Neural Networks, Vol. 1. pp. 75-89, 1988,and "Learned Classification of Sonar Targets Using a Massively ParallelNetwork", IEEE Transactions on Acoustics, Speech, and Signal Processing,Vol. 36, No. 7, July 1988.

"Pattern recognition" as used herein will mean any system whichprocesses a signal that is generated by an object, or is modified byinteracting with an object, in order to determine which one of a set ofclasses that the object belongs to. Such a system might determine onlythat the object is or is not a member of one specified class, or itmight attempt to assign the object or one of a larger set of specifiedclasses, or find that it is not a member of any of the classes in theset. The signals processed are generally electrical signals coming fromtransducers which are sensitive to either acoustic or electromagneticradiation and if electromagnetic, they can be either visible light,infrared, ultraviolet or radar.

"To identify" as used herein will mean to determine that the objectbelongs to a particular set or class. The class may be containing allrear facing child seats, one containing all human occupants, or allhuman occupants not sitting in a rear facing child seat depending on thepurpose of the system. In the case where a particular person is to berecognized, the set or class will contain only a single element, theperson to be recognized.

Some examples follow:

In a passive infrared system a detector receives infrared radiation froman object in its field of view, in this case the vehicle occupant, anddetermines the temperature of the occupant based on the infraredradiation. The VIMS can then respond to the temperature of the occupant,which can either be a child in a rear facing child seat or a normallyseated occupant, to control some other system. This technology couldprovide input data to a pattern recognition system but it haslimitations related to temperature. The sensing of the child could posea problem if the child is covered with blankets. It also might not bepossible to differentiate between a rear facing child seat and a forwardfacing child seat. In all cases, the technology will fail to detect theoccupant if the ambient temperature reaches body temperature as it doesin hot climates. Nevertheless, for use in the control of the vehicleclimate, for example, a passive infrared system that permits an accuratemeasurement of each occupant's temperature is useful.

In a laser optical system an infrared laser beam is used to momentarilyilluminate an object, occupant or child seat in the manner as described,and illustrated in FIG. 8, of U.S. Pat. No. 5,653,462 cross referencedabove. In some cases, a charge-coupled device (a type of TV camera alsoreferred to as a CCD array) or a CMOS device is used to receive thereflected light. The laser can either be used in a scanning mode, or,through the use of a lens, a cone of light can be created which covers alarge portion of the object. Also triangulation can be used inconjunction with an offset scanning laser to determine the range of theilluminated spot from the light detector. In each case, a patternrecognition system, as defined above, is used to identify and classify,and can be used to locate, the illuminated object and its constituentparts. This system provides the most information about the object and ata rapid data rate. Its main drawback is cost which is considerably abovethat of ultrasonic or passive infrared systems. As the cost of laserscomes down in the future, this system will become more competitive.Depending on the implementation of the system, there may be some concernfor the safety of the occupant if the laser light can enter theoccupant's eyes. This is minimized if the laser operates in the infraredspectrum.

Radar systems have similar properties to the laser system discussedabove. The wavelength of a particular radar system can limit the abilityof the pattern recognition system to detect object features smaller thana certain size. Once again, however, there is some concern about thehealth effects of radar on children and other occupants. This concern isexpressed in various reports available from the United States Food andDrug Administration Division of Devices.

The ultrasonic system is the least expensive and potentially providesless information than the laser or radar systems due to the delaysresulting from the speed of sound and due to the wave length which isconsiderably longer than the laser systems. The wavelength limits thedetail, which can be seen by the system. In spite of these limitations,as shown below, ultrasonics can provide sufficient timely information topermit the position and velocity of an occupant to be accurately knownand, when used with an appropriate pattern recognition system, it iscapable of positively determining the presence of a rear facing childseat. One pattern recognition system which has been used to identify arear facing child seat uses neural networks and is similar to thatdescribed in the above referenced papers by Gorman et al.

A focusing system, such as used on camera systems, could be used todetermine the initial position of an occupant but is too slow to monitorhis position during a crash. This is a result of the mechanical motionsrequired to operate the lens focusing system. By itself it cannotdetermine the presence of a rear facing child seat or of an occupant butwhen used with a charge-coupled device plus some infrared illuminationfor night vision, and an appropriate pattern recognition system, thisbecomes possible.

From the above discussion, it can be seen that the addition ofsophisticated pattern recognition means to any of the standardillumination and/or reception technologies for use in a motor vehiclepermits the development of a host of new products, systems orcapabilities heretofore not available and as described in more detailbelow.

OBJECTS AND SUMMARY OF THE INVENTION

This invention is a system to sense the presence, position and type ofan occupant in a passenger compartment of a motor vehicle and moreparticularly, to identify and monitor occupants and their parts andother objects in the passenger compartment of a motor vehicle, such asan automobile or truck, by processing one or more signals received fromthe occupants and their parts and other objects using one or more of avariety of pattern recognition techniques and illumination technologies.The received signal(s) may be a reflection of a transmitted signal, thereflection of some natural signal within the vehicle, or may be somesignal emitted naturally by the object. Information obtained by theidentification and monitoring system is then used to affect theoperation of some other system in the vehicle.

The principle objects and advantages are:

1. To recognize the presence of a human on a particular seat of a motorvehicle and to use this information to affect the operation of anothervehicle system such as the airbag system, heating and air conditioningsystem, or entertainment system, among others.

2. To recognize the presence of a human on a particular seat of a motorvehicle and then to determine his/her position and to use this positioninformation to affect the operation of another vehicle system.

3. To recognize the presence of a human on a particular seat of a motorvehicle and then to determine his/her velocity relative to the passengercompartment and to use this velocity information to affect the operationof another vehicle system.

4. To determine the position of a seat in the vehicle using sensorsremote from the seat and to use that information in conjunction with amemory system and appropriate actuators to position the seat to apredetermined location.

5. To determine the position, velocity or size of an occupant in a motorvehicle and to utilize this information to control the rate of gasgeneration, or the amount of gas generated, by an airbag inflator systemor otherwise control the flow of gas into or out of an airbag.

6. To determine the fact that an occupant is not restrained by aseatbelt and therefore to modify the characteristics of the airbagsystem. This determination can be done either by monitoring the positionof the occupant or through the use of a resonating device placed on theshoulder belt portion of the seatbelt.

7. To determine the presence or position of rear seated occupants in thevehicle and to use this information to affect the operation of a rearseat protection airbag for frontal impacts.

8. To determine the presence or position of occupants relative to theside impact airbag systems and to use this information to affect theoperation of a side impact protection airbag system.

9. To determine the openness of a vehicle window and to use thatinformation to affect another vehicle system.

10. To determine the presence of an occupant's hand or other object inthe path of a closing window and to affect the window closing system.

11. To remotely determine the fact that a vehicle door is not tightlyclosed using an illumination transmitting and receiving system such asone employing electromagnetic or acoustic waves.

12. To determine the position of the shoulder of a vehicle occupant andto use that information to control the seatbelt anchorage point.

13. To determine the position of the rear of an occupant's head and touse the information to control the position of the headrest.

14. To recognize the presence of a rear facing child seat on aparticular seat of a motor vehicle and to use this information to affectthe operation of another vehicle system such as the airbag system.

15. To determine the total number of occupants of a vehicle and in theevent of an accident to transmit that information, as well as otherinformation such as the condition of the occupants, to a receiver remotefrom the vehicle.

16. To affect the vehicle entertainment system based on a determinationof the size or location of various occupants or other objects within thevehicle passenger compartment.

17. To affect the vehicle heating, ventilation and air conditioningsystem based on a determination of the number, size and location ofvarious occupants or other objects within the vehicle passengercompartment.

18. To determine the temperature of an occupant based on infraredradiation coming from that occupant and to use that information tocontrol the heating, ventilation and air conditioning system.

19. To provide a vehicle interior monitoring system for determining thelocation of occupants within the vehicle and to include within the samesystem various electronics for controlling an airbag system.

20. To determine the approximate location of the eyes of a driver and touse that information to control the position of the rear view mirrors ofthe vehicle.

21. To monitor the position of the head of the vehicle driver anddetermine whether the driver is falling asleep or otherwise impaired andlikely to lose control of the vehicle and to use that information toaffect another vehicle system.

22. To monitor the position of the eyelids of the vehicle driver anddetermine whether the driver is falling asleep or otherwise impaired andlikely to lose control of the vehicle and to use that information toaffect another vehicle system.

23. To determine the location of the eyes of a vehicle occupant and thedirection of a light source such as the headlights of an oncomingvehicle or the sun and to cause a filter to be placed in such a manneras to reduce the intensity of the light striking the eyes of theoccupant.

24. To determine the location of the eyes of a vehicle occupant and thedirection of a light source such as the headlights of a rear approachingvehicle or the sun and to cause a filter to be placed in such a mannerto reduce the intensity of the light reflected from the rear viewmirrors and striking the eyes of the occupant.

25. To determine the location of the ears of a vehicle occupant and touse that information to control the entertainment system so as toimprove the quality of the sound reaching the occupant's ears throughsuch methods as noise canceling sound.

26. To recognize a particular driver based on such factors as physicalappearance or other attributes and to use this information to controlanother vehicle system such as a security system, seat adjustment, ormaximum permitted vehicle velocity, among others.

These and other objects and advantages will become apparent from thefollowing description of the preferred embodiments of the vehicleinterior identification and monitoring system of this invention andmethod and system for optimizing the comfort of a vehicular occupant.

The system for optimizing the comfort of the occupant in accordance withthe invention therefore comprises first determining means fordetermining the direction of a bothersome stimulus, such as an wantedsound, external or internal of the vehicle, and an external lightsource, second determining means for determining the location of thesensory organ of the occupant receptive to the stimulus, e.g., the earsof the occupant if the stimulus is an unwanted sound and the eyes of theoccupant if the stimulus is an external light source, and activationmeans coupled to the first and second determining means responsive tothe determined direction of the stimulus and the determined location ofthe sensory organ of the occupant receptive thereto for automaticallyactivating a countereffect to the stimulus in order to diminish theeffect of the stimulus on the sensory organ of the occupant. If thestimulus is the external light source, the activation means may bestructured and arranged to activate a light filter between the externallight source and the eyes of the occupant. The light filter may compriseelectro-chromic glass and/or liquid crystal film arranged in connectionwith windows of the vehicle, e.g., the windshield and/or the rear-viewmirror. The filter may be fabricated to limit the amount of lightpassing through selected portions of the windshield or rear view mirror.

If the stimulus is a source of unwanted sound, the location of the earsof the occupant is determined and the activation means comprise soundgeneration and transmission means which generate and transmit soundwaves to cancel the unwanted sound at the ears of the occupant. Thestimulus may be both sound and a light source, in which case, additionaldetermining means and activation means are provided.

The determining means which determine the location of the sensory organof the occupant, i.e., his eyes or ears, comprise a transmitter fortransmitting waves toward the occupant, a plurality of receivers forreceiving waves reflected from the occupant and a processor forprocessing the received reflected waves into an indication of thelocation of the sensory organ of the occupant based in part on patternrecognition techniques. A multitude of different occupant locationdetermining means are disclosed herein in conjunction with a vehicleinterior monitoring and identification system and any of the disclosedsystems may be used in conjunction with he comfort optimizing system andmethods.

The method for optimizing the comfort of a vehicular occupant comprisesthe steps of determining the direction of a bothersome stimulus, i.e.,an unwanted sound of source light, determining the location of thesensory organ of the occupant receptive to the stimulus, i.e., the earsor eyes of the occupant, and in consideration of the determineddirection of the stimulus and the determined location of the sensoryorgan of the occupant receptive to the stimulus, automaticallyactivating a countereffect to the stimulus in order to diminish theeffect of the stimulus on the sensory organ of the occupant, i.e.,activating a light filter between the external light source and the eyesof the occupant or generating and transmitting sound waves to cancel theunwanted sound at the ears of the occupant.

Furthermore, the present invention also relates to methods forcontrolling a system in the vehicle based on an occupying item in whichat least a portion of the passenger compartment in which the occupyingitem is situated is irradiated, reflected radiation from surfaces of theoccupying item are received, e.g., by a plurality of sensors ortransducers each arranged at a discrete location, the received radiationis processed by a processor in order to create one or more electronicsignals characteristic of the occupying item based on the receivedradiation, each signal containing a pattern representative and/orcharacteristic of the occupying item and each signal is then categorizedby utilizing pattern recognition means for recognizing and thusidentifying the class of the occupying item. The pattern recognitionmeans process each signal into a categorization thereof based on datacorresponding to patterns of received radiation stored within thepattern recognition means and associated with possible classes ofoccupying items of the vehicle. Once the signal(s) is/are categorized,the operation of the system in the vehicle may be affected based on thecategorization of the signal(s), and thus based on the occupying item.

If the system in the vehicle is a vehicle communication system, then anoutput representative of the number of occupants in the vehicle may beproduced based on the categorization of the signal(s) and the vehiclecommunication system thus controlled based on such output. Similarly, ifthe system in the vehicle is a vehicle entertainment system or heatingand air conditioning system, then an output representative of specificseat occupancy may be produced based on the categorization of thesignal(s) and the vehicle entertainment or heating and air conditioningsystem thus controlled based on such output.

In one embodiment designed to ensure safe operation of the vehicle, theattentiveness of the occupying item is determined from the signal(s) ifthe occupying item is an occupant, and in addition to affecting thesystem in the vehicle based on the categorization of the signal, thesystem in the vehicle is affected based on the determined attentivenessof the occupant.

One embodiment of the interior monitoring system in accordance with theinvention comprises means for irradiating at least a portion of thepassenger compartment in which an occupying item is situated, receivermeans for receiving reflected radiation from the occupying item, e.g., aplurality of receivers, each arranged at a discrete location, processormeans coupled to the receivers for processing the received radiationfrom each receiver in order to create a respective electronic signalcharacteristic of the occupying item based on the received radiation,each signal containing a pattern representative of the occupying item,categorization means coupled to the processor means for categorizing thesignals, and output means coupled to the categorization means foraffecting at least one other system within the vehicle based on thecategorization of the signals characteristic of the occupying item. Thecategorization means may comprise pattern recognition means forrecognizing and thus identifying the class of the occupying item byprocessing the signals into a categorization thereof based on datacorresponding to patterns of received radiation stored within thepattern recognition means and associated with possible classes ofoccupying items of the vehicle. Each signal may comprise a plurality ofdata, all of which is compared to the data corresponding to patterns ofreceived radiation stored within the pattern recognition means andassociated with possible classes of contents of the vehicle.

In one specific embodiment, the system includes location determiningmeans coupled to the processor means for determining the location of theoccupying item, e.g., based on the received radiation such that theoutput means which are coupled to the location determining means, inaddition to affecting the other system based on the categorization ofthe signals characteristic of the occupying item, affect the systembased on the determined location of the occupying item.

In another embodiment to determine the presence or absence of anoccupant, the categorization means comprise pattern recognition meansfor recognizing the presence or absence of an occupying item in thepassenger compartment by processing each signal into a categorizationthereof signal based on data corresponding to patterns of receivedradiation stored within the pattern recognition means and associatedwith possible occupying items of the vehicle and the absence of suchoccupying items.

All of the above-described methods and apparatus may be used inconjunction with one another and in combination with the methods andapparatus for optimizing the driving conditions for the occupants of thevehicle described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the followingnon-limiting drawings.

FIG. 1 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a rear facing child on thefront passenger seat and a preferred mounting location for an occupantand rear facing child seat presence detector.

FIG. 1A is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a rear facing child seat onthe front passenger seat having a resonator or reflector placed onto theforward most portion of the child seat.

FIG. 2 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle cellular communication system.

FIG. 3 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle heating and air conditioningsystem.

FIG. 4 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle airbag system.

FIG. 5 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle entertainment system.

FIG. 6 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a driver and a preferredmounting location for an occupant identification system.

FIG. 7A is a functional block diagram of the ultrasonic imaging systemillustrated in FIG. 1 using a microprocessor.

FIG. 7B is a functional block diagram of the ultrasonic imaging systemillustrated in FIG. 1 using an application specific integrated circuit(ASIC).

FIG. 8 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a box on the frontpassenger seat and a preferred mounting location for an occupant andrear facing child seat presence detector.

FIG. 9 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a driver and a preferredmounting location for an occupant position sensor for use in sideimpacts and also of a rear of occupant's head locator for use with aheadrest adjustment system to reduce whiplash injuries in rear impactcrashes.

FIG. 10 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a front passenger and apreferred mounting location for an occupant head detector and apreferred mounting location of an adjustable microphone and speakers.

FIG. 11 is a side view with parts cutaway and removed of a subjectvehicle and an oncoming vehicle, showing the headlights of the oncomingvehicle and the passenger compartment of the subject vehicle, containingdetectors of the driver's eyes and detectors for the headlights of theoncoming vehicle and the selective filtering of the light of theapproaching vehicle's headlights through the use of electro-chromicglass in the windshield.

FIG. 11A is an enlarged view of the section designated A in FIG. 11.

FIG. 12 is a side view with parts cutaway and removed of a vehicle and afollowing vehicle showing the headlights of the following vehicle andthe passenger compartment of the leading vehicle containing a driver anda preferred mounting location for driver eyes and following vehicleheadlight detectors and the selective filtering of the light of thefollowing vehicle's headlights through the use of electro-chromic glassin the rear view mirror.

FIG. 12A is an enlarged view of the section designated A in FIG. 12.

FIG. 13 is a side view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a driver, a shoulder heightsensor and a seatbelt anchorage adjustment system.

FIG. 14 is a side view with parts cutaway and removed of a seat in thepassenger compartment of a vehicle showing the use of ultrasonicresonators or reflectors to determine the position of the seat.

FIG. 15 is a side view with parts cutaway and removed of the passengercompartment of a vehicle showing the use of ultrasonic resonators orreflectors to determine the position of the driver seatbelt.

FIG. 16 is a side view with parts cutaway and removed of the passengercompartment of a vehicle showing the use of an ultrasonic resonator orreflector to determine the extent of opening of the driver window and ofa system for determining the presence of an object, such as the hand ofan occupant, in the window opening.

FIG. 16A is a side view with parts cutaway and removed of the passengercompartment of a vehicle showing the use of an ultrasonic resonator orreflectors to determine the extent of opening of the driver window andof another system for determining the presence of an object, such as thehand of an occupant, in the window opening.

FIG. 17 is a side view with parts cutaway and removed of the passengercompartment of a vehicle showing the use of an ultrasonic resonator orreflectors to determine the extent of opening of the driver side door.

FIG. 18 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle security system.

FIG. 19 is a side view with parts cutaway and removed showingschematically the interface between the vehicle interior monitoringsystem of this invention and an instrument panel mounted inattentivenesswarning light or buzzer and reset button.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to the accompanying drawings wherein the same referencenumerals refer to the same or similar elements, in FIG. 1 a side view,with parts cutaway and removed, of a vehicle showing the passengercompartment containing a rear facing child seat 110 on a front passengerseat 120 and a preferred mounting location for an occupant and rearfacing child seat presence detector in accordance with the invention isillustrated. In this implementation, three ultrasonic transducers 131,132 and 133 are used in the presence detector in accordance with theinvention, although any number of wave-transmitting transducers orradiation-receiving receivers may be used. Such transducers or receiversmay be of the type which emit or receive a continuous signal, a timevarying signal or a spacial varying signal such as in a scanning system.Transducer 132 transmits ultrasonic energy toward the front passengerseat, which is reflected, in this case by the occupying item of thepassenger seat, i.e., the rear facing child seat 110, and the reflectedwaves are received by the transducers 131 and 133. The waves received bytransducers 131 and 133 vary with time depending on the shape of theobject occupying the passenger seat, in this case the rear facing childseat 110. Each object will reflect back a signal having a differentpattern. Also, the pattern received by transducer 131 will differslightly from the pattern received by transducer 133 in view of a itsdifferent mounting location. In some systems, this difference permitsthe determination of location of the reflecting surface throughtriangulation. Through the use of two transducers 131,133, a sort ofstereographic image is received by the two transducers and recorded foranalysis by processor 101, which is coupled to the transducers131,132,133. This image will differ for each object that is placed onthe vehicle seat and it will also change for each position of aparticular object and for each position of the vehicle seat.

The "image" recorded from each ultrasonic transducer/receiver, forultrasonic systems, is actually a time series of digitized data of theamplitude of the received signal versus time. Since there are tworeceivers, two time series are obtained which are processed by theprocessor 101. When different objects are placed on the front passengerseat the two images are different but there are also similaritiesbetween all images of rear facing child seats, for example, regardlessof where on the vehicle seat it is placed and regardless of what companymanufactured the child seat. Alternately, there will be similaritiesbetween all images of people sitting on the seat regardless of what theyare wearing, their age or size. The problem is to find the "rules" whichdifferentiate the occupant images from the rear facing child seatimages. The similarities of these images for various child seats arefrequently not obvious to a person looking at plots of the time seriesand thus computer algorithms are developed to sort out the variouspatterns.

The determination of these rules is central to the pattern recognitiontechniques used in this invention. In general, three approaches havebeen useful, artificial intelligence, fuzzy logic and artificial neuralnetworks. In some implementations of this invention, such as thedetermination that there is an object in the path of a closing window asdescribed below, the rules are sufficiently obvious that a trainedresearcher can look at the returned acoustic signals and devise a simplealgorithm to make the required determinations. In others, such as thedetermination of the presence of a rear facing child seat or of anoccupant, artificial neural networks are used to determine the rules.One such set of neural network software for determining the patternrecognition rules is available from the NeuralWare Corporation ofPittsburgh, Pa.

The system used in a preferred implementation of this invention for thedetermination of the presence of a rear facing child seat, of anoccupant or of an empty seat was the artificial neural network. In thiscase, the network operates on the two returned signals as sensed bytransducers 131 and 133. Through a training session, the system istaught to differentiate between the three cases. This is done byconducting a large number of experiments where all possible child seatsare placed in all possible orientations on the front passenger seat.Similarly a sufficiently large number of experiments are run with humanoccupants and with boxes, bags of groceries and other objects. Sometimesas many as 1,000,000 such experiments are run before the neural networkis sufficiently trained so that it can differentiate among the threecases and output the correct decision with a very high probability.

Once the network is determined, it is possible to examine the resultusing tools supplied by NeuralWare, for example, to determine the rulesthat were finally arrived at by the trial and error techniques. In thatcase, the rules can then be programmed into a microprocessor resultingin a fuzzy logic or other rule based system. Alternately, a neuralcomputer can be used to implement the net directly. In either case, theimplementation can be carried out by those skilled in the art of patternrecognition. If a microprocessor is used, a memory device is alsorequired to store the data from the analog to digital converters thatdigitize the data from the receiving transducers. On the other hand, ifa neural network computer is used, the analog signal can be fed directlyfrom the transducers to the neural network input nodes and anintermediate memory is not required. Memory of some type is needed tostore the computer programs in the case of the microprocessor system andif the neural computer is used for more than one task, a memory isneeded to store the network specific values associated with each task.

An alternate system is shown in FIG. 2, which is a side view showingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle cellular communication system.In this view, an adult occupant 210 is shown sitting on the frontpassenger seat 220 and two ultrasonic transducers 231 and 232 are usedto determine the presence (or absence) of the occupant on that seat 220.One of the transducers 232 in this case acts as both a transmitter andreceiver while transducer 231 acts only as a receiver. Alternately,transducer 231 could serve as both a transmitter and receiver or thetransmitting function could be alternated between the two devices. Also,in many cases more that two transmitters and receivers are used and instill other cases other types of sensors, such as weight, seatbelt andseat position sensors, are also used in combination with the radiationsensors. As was also the case in FIG. 1, the transducers 231 and 232 areattached to the vehicle buried in the A-pillar trim, where theirpresence is disguised, and are connected to processor 101 that is alsohidden in the trim. The A-pillar is the roof support pillar that isclosest to the front of the vehicle and which, in addition to supportingthe roof, also supports the front windshield and the front door.Naturally, other mounting locations can also be used.

The interface between the monitoring system and the cellular phonesystem is shown schematically by box 240 that outputs to an antenna250A. The transducers 231 and 232 in conjunction with the patterrecognition hardware and software, which is implemented in processor 101and is packaged on a printed circuit board or flex circuit along withthe transducers 231 and 232, determine the presence of an occupantwithin a few seconds after the vehicle is started. Similar systemslocated to monitor the remaining seats in the vehicle, also determinethe presence of occupants at the other seating locations and this resultis stored in the computer memory which is part of each monitoring systemprocessor 101. In the event of an accident, the electronic systemassociated with the cellular phone system interrogates the variousinterior monitoring system memories and arrives at a count of the numberof occupants in the vehicle, and in more sophisticated systems , evenmakes a determination as to whether each occupant was wearing a seatbeltand if he or she is moving after the accident. The phone system thenautomatically dials the EMS operator (such as 911) and the informationobtained from the interior monitoring systems if forwarded so that adetermination can be made as to the number of ambulances and otherequipment to send to the accident site. Vehicles having this capabilityare now in service. Such vehicles will also have a system, such as theglobal positioning system, which permits the vehicle to determine itsexact location and to forward this information to the EMS operator.

The control of the heating, ventilating, and air conditioning (HVAC)system alone would probably not justify the implementation of aninterior monitoring system at least until the time comes when electronicheating and cooling systems replace the conventional systems now used.Nevertheless, if the monitoring system is present, it can be used tocontrol the HVAC for a small increment in cost. The advantage of such asystem is that since most vehicles contain only a single occupant, thereis no need to direct heat or air conditioning to unoccupied seats. Thispermits the most rapid heating or cooling for the driver when thevehicle is first started and he is alone without heating or coolingunoccupied seats. Since the HVAC system does consume energy, an energysaving also results by only heating and cooling the driver when he isalone.

FIG. 3 shows a side view of a vehicle passenger compartment showingschematically an interface 260 between the vehicle interior monitoringsystem of this invention and the vehicle heating and air conditioningsystem. In addition to the transducers 231 and 232, which at least inthis embodiment are preferably acoustic transducers, an infrared sensor234 is also shown mounted in the A-pillar and which monitors thetemperature of the occupant. The output from each of the transducers isfed into processor 101 that is in turn connected to interface 260. Inthis manner, the HVAC control is based on the occupant's temperaturerather than that of the ambient air in the vehicle, as well as thedetermined presence of the occupant via transducers 231,232 as describedabove. This also permits each vehicle occupant to be independentlymonitored and the HVAC system to be adjusted for each occupant eitherbased on a set temperature for all occupants or, alternately, eachoccupant could be permitted to set his own preferred temperature throughadjusting a control knob shown schematically as 250 in FIG. 3. Since themonitoring system is already installed in the vehicle with its ownassociated electronics including processor 101, the infrared sensor canbe added with little additional cost and can share the processing unit.Not only can this system be used for directing hot and cold air, butdevelopments in the field of directing sound using hyper-sound now makesit possible to accurately direct sound to the vicinity of the ears of anoccupant so that only that occupant can hear the sound. The system ofthis invention can thus be used to find the proximate direction of theears of the occupant for this purpose.

The use of the vehicle interior monitoring system to control thedeployment of an airbag is discussed in detail in U.S. Pat. No.5,653,462 cross referenced above. In that case, the control is based onthe use of a simple pattern recognition system to differentiate betweenthe occupant and his extremities in order to provide an accuratedetermination of the position of the occupant relative to the airbag. Ifthe occupant is sufficiently close to the airbag module that he is morelikely to be injured by the deployment itself than by the accident, thedeployment of the airbag is suppressed. This process is carried furtherby the interior monitoring system described herein in that the nature oridentity of the object occupying the vehicle seat is used to contributeto the airbag deployment decision. FIG. 4 shows a side view illustratingschematically the interface between the vehicle interior monitoringsystem of this invention and the vehicle airbag system 270.

In this embodiment, an ultrasonic transducer 232 transmits a burst ofultrasonic waves that travel to the occupant where they are reflectedback to transducers or receptors/receivers 231 and 232. The time periodrequired for the waves to travel from the generator and return is usedto determine the distance from the occupant to the airbag as describedin the aforementioned U.S. Pat. No. 5,653,462, i.e., and thus may alsobe used to determine the position or location of the occupant. In thecase of this invention, however, the portion of the return signal, whichrepresents the occupants' head or chest, has been determined based onpattern recognition techniques such as a neural network. The relativevelocity of the occupant toward the airbag can then be determined, fromsuccessive position measurements, which permits a sufficiently accurateprediction of the time when the occupant would become proximate to theairbag. By comparing the occupant relative velocity to the integral ofthe crash deceleration pulse, a determination as to whether the occupantis being restrained by a seatbelt can also be made which then can affectthe airbag deployment initiation decision. Alternately, the mereknowledge that the occupant has moved a distance which would not bepossible if he were wearing a seatbelt gives information that he is notwearing one.

A more detailed discussion of this process and of the advantages of thevarious technologies, such as acoustic or electromagnetic, can be foundin SAE paper 940527, "Vehicle Occupant Position Sensing" by Breed et al,which is included herein by reference. In this paper, it is demonstratedthat the time delay required for acoustic waves to travel to theoccupant and return does not prevent the use of acoustics for positionmeasurement of occupants during the crash event. For positionmeasurement and for many pattern recognition applications, ultrasonicsis the preferred technology due to the lack of adverse health effectsand the low cost of ultrasonic systems compared with either laser orradar. The main limiting feature of ultrasonics is the wavelength, whichplaces a limitation of the size of features that can be discerned.Optical systems, for example, are required when the identification ofparticular individuals is required.

It is well known among acoustics engineers that the quality of soundcoming from an entertainment system can be substantially affected by thecharacteristics and contents of the space in which it operates and thesurfaces surrounding that space. When an engineer is designing a systemfor an automobile he has a great deal of knowledge about that space andof the vehicle surfaces surrounding it. He has little knowledge of howmany occupants are likely to be in the vehicle on a particular day,however, and therefore the system is a compromise. If the system knewthe number and position of the vehicle occupants, and maybe even theirsize, then adjustments could be made in the system output and the soundquality improved. FIG. 5, therefore, illustrates schematically theinterface between the vehicle interior monitoring system of thisinvention, i.e., transducers 231,232 and processor 101 which operates asset forth above, and the vehicle entertainment system 280. Theparticular design of the entertainment system that uses the informationprovided by the monitoring system can be determined by those skilled inthe appropriate art. Perhaps in combination with this system, thequality of the sound system can be measured by the audio system itselfeither by using the speakers as receiving units also or through the useof special microphones. The quality of the sound can then be adjustedaccording to the vehicle occupancy and the reflectivity of the vehicleoccupants. If, for example, certain frequencies are being reflected morethat others, the audio amplifier can be adjusted to amplify thosefrequencies to a lesser amount that others.

Vehicle entertainment system 280 may include means for generating andtransmitting sound waves at the ears of the occupants, the position ofwhich are detected by transducers 231,232 and processor 101, as well asmeans for detecting the presence and direction of unwanted noise. Inthis manner, appropriate sound waves can be generated and transmitted tothe occupant to cancel the unwanted noise and thereby optimize thecomfort of the occupant, i.e., the reception of the desired sound fromthe entertainment system 280.

The maximum acoustic frequency that is practical to use for acousticimaging in the systems is about 40 to 160 kilohertz (kHz). Thewavelength of a 50 kHz acoustic wave is about 0.6 cm which is too coarseto determine the fine features of a person's face, for example. It iswell understood by those skilled in the art that features which aresmaller than the wavelength of the illuminating radiation cannot bedistinguished. Similarly the wave length of common radar systems variesfrom about 0.9 cm (for 33,000 MHz K band) to 133 cm (for 255 MHz P band)which is also too coarse for person identification systems. In FIG. 6,therefore, the ultrasonic transducers of the previous designs arereplaced by laser transducers 231 and 232 which are connected to amicroprocessor 101. In all other manners, the system operates the same.The design of the electronic circuits for this laser system is describedin some detail in the U.S. Pat. No. 5,653,462 cross-referenced above andin particular FIG. 8 thereof and the corresponding description. In thiscase, a pattern recognition system such as a neural network system isemployed and uses the demodulated signals from the receptors 231 and232.

The output of processor 101 of the monitoring system is shown connectedschematically to a general interface 290 which can be the vehicleignition enabling system; the entertainment system; the seat, mirror,suspension or other adjustment systems; or any other appropriate vehiclesystem.

There are two preferred methods of implementing the vehicle interiormonitoring system of this invention, a microprocessor system and anapplication specific integrated circuit system (ASIC). Both of thesesystems are represented schematically as either 101 or 601 herein. Insome systems, both a microprocessor and an ASIC are used. In othersystems, most if not all of the circuitry is combined onto a single chip(system on a chip). The particular implementation depends on thequantity to be made and economic considerations. A block diagramillustrating the microprocessor system is shown in FIG. 7A which showsthe implementation of the system of FIG. 1. An alternate implementationof the FIG. 1 system using an ASIC is shown in FIG. 7B. In both casesthe target, which may be a rear facing child seat, is shownschematically as 110 and the three transducers as 131, 132, and 133. Inthe embodiment of FIG. 7A, there is a digitizer coupled to the receivers131,133 and the processor, and an indicator coupled to the processor. Inthe embodiment of FIG. 7B, there is memory unit associated with the ASICand also an indicator coupled to the ASIC.

In FIG. 8, a view of the system of FIG. 1 is illustrated with a box 295shown on the front passenger seat in place of the rear facing childseat. The vehicle interior monitoring system of this invention istrained to recognize that this box 295 is neither a rear facing childseat nor an occupant and therefore it is treated as an empty seat andthe deployment of the airbag is suppressed. This training isaccomplished using a neural network with the commercially availablesoftware disclosed above and provided, for example, by NeuralWare ofPittsburgh. The system accesses the probability that the box is aperson, however, and if there is even the remotest chance that it is aperson, the airbag deployment is not suppressed. The system is thustypically biased toward enabling airbag deployment.

Side impact airbags are now beginning to be used on some vehicles. Theseinitial airbags are quite small compared to the driver or passengerairbags used for frontal impact protection. Nevertheless, a small childcould be injured if he is sleeping with his head against the airbag whenit deploys and a vehicle interior monitoring system is needed to preventsuch a deployment in that event. In FIG. 9, a single ultrasonictransducer 330 is shown mounted in the vehicle door adjacent to theairbag system. This device is not used to identify the object that isadjacent the airbag but merely to measure the position of the object.

A rear-of-head detector 334 is also illustrated in FIG. 9. This detector334 is used to determine the distance from the headrest to the rear mostposition of the occupant's head and to therefore control the position ofthe headrest so that it is properly positioned behind the occupant'shead to offer optimum support in the event of a rear impact. Althoughthe headrest of most vehicles is adjustable, it is rare for an occupantto position it properly if at all. Each year there are in excess of400,000 whiplash injuries in vehicle impacts approximately 90,000 ofwhich are from rear impacts (source: National Highway Traffic SafetyAdministration, (NHTSA)).A properly positioned headrest couldsubstantially reduce the frequency of such injuries, which can beaccomplished by the head detector of this invention. The head detector334 is shown connected schematically to the headrest control mechanismand circuitry 340. This mechanism is capable of moving the headrest upand down and, in some cases, rotating it fore and aft.

When the driver of a vehicle is using a cellular phone, the phonemicrophone frequently picks up other noise in the vehicle making itdifficult for the other party to hear what is being said. This noise canbe reduced if a directional microphone is used and directed toward themouth of the driver. This is difficult to do since the position ofdrivers' mouths varies significantly depending on such things as thesize and seating position of the driver. By using the vehicle interioridentification and monitoring system of this invention, and throughappropriate pattern recognition techniques, the location of the driver'shead can be determined with sufficient accuracy even with ultrasonics topermit a directional microphone having a 15 degree cone angle to beaimed at the mouth of the driver resulting in a clear reception of hisvoice. The use of directional speakers in a similar manner also improvesthe telephone system performance. In the extreme case of directionality,the techniques of hypersound can be used. Such a system can also be usedto permit effortless conversations between occupants of the front andrear seats. Such a system is shown in FIG. 10 which is a system similarto that of FIG. 2 only using three ultrasonic transducers 231, 232 and233 to determine the location of the driver's head and control thepointing direction of a microphone 355. Speaker 357 is shown connectedschematically to the phone system 359 completing the system.

The transducers 231 and 232 are placed high in the A-pillar and thethird transducer 233 is placed in the headliner and displacedhorizontally from transducers 231 and 232. The two transducers 231 and232 provide information to permit the determination of the locus of thehead in the vertical direction and the combination of one of transducers231 and 232 in conjunction with transducers 233 is used to determine thehorizontal location of the head. The three transducers are placed highin the vehicle passenger compartment so that the first returned signalis from the head. Temporal filtering is used to eliminate signals whichare reflections from beyond the head and the determination of the headcenter location is then found by the approximate centroid of the headreturned signal. That is, once the location of the return signalcentroid is found from each of the three received signals fromtransducers 231, 232 and 233, the distance to that point is known foreach of the transducers based on the time it takes the signal to travelfrom the head to each transducer. In this manner, by using the threetransducers plus an algorithm for finding the coordinates of the headcenter, using processor 101, and through the use of known relationshipsbetween the location of the mouth and the head center, an estimate ofthe mouth location, and the ear locations, can be easily determinedwithin a circle having a diameter of about five inches (13 cm). This issufficiently accurate for a directional microphone to cover the mouthwhile excluding the majority of unwanted noise.

The headlights of oncoming vehicles frequently make it difficult for thedriver of a vehicle to see the road and safely operate the vehicle. Thisis a significant cause of accidents and much discomfort. The problem isespecially severe during bad weather where rain can cause multiplereflections. Visors are now used to partially solve this problem butthey do so by completely blocking the view through a large portion ofthe window and therefore cannot be used to cover the entire windshield.Similar problems happen when the sun is setting or rising and the driveris operating the vehicle in the direction of the sun. The vehicleinterior monitoring system of this invention can contribute to thesolution of this problem by determining the position of the driver'seyes. If separate sensors are used to sense the direction of the lightfrom the on-coming vehicle or the sun, and through the use ofelectro-chromic glass, a liquid crystal film, or other appropriatetechnology, a portion of the windshield can be darkened to impose afilter between the eyes of the driver and the light source.Electro-chromic glass is a material where the color of the glass can bechanged through the application of an electric mirror. By dividing thewindshield into a controlled grid or matrix of contiguous areas andthrough feeding the current into the windshield from orthogonaldirections, selective portions of the windshield can be darkened asdesired. Other systems for selectively imposing a filter between theeyes of an occupant and the light source are currently underdevelopment.

FIG. 11 illustrates how such a system operates. A sensor 410 located onvehicle 402 determines the direction of the light 412 from theheadlights of oncoming vehicle 404. Sensor 410 is comprised of a lensand a charge-coupled device (CCD), of CMOS light sensing or similardevice, with appropriate electronic circuitry which determines whichelements of the CCD are being most brightly illuminated. An algorithmstored in processor 101 then calculates the direction of the light fromthe oncoming headlights based on the information from the CCD, or CMOSdevice. Transducers 231, 232 and 233 determine the probable location ofthe eyes of the operator 210 of vehicle 402 in a manner such asdescribed above in conjunction with the determination of the location ofthe driver's mouth in the discussion of FIG. 10. In this case, thedetermination of the probable locus of the driver's eyes is made with anaccuracy of a diameter for each eye of about 3 inches (7.5 cm). Thiscalculation sometimes will be in error and provision is made for thedriver to make an adjustment to correct for this error as describedbelow.

The windshield 416 of vehicle 402 is made from electro-chromic glass orcomprises a liquid crystal film or similar system, and is selectivelydarkened at area 418 due to the application of a current alongperpendicular directions 422 and 424 of windshield 416. The particularportion of the windshield to be darkened is determined by processor 101.Once the duration of the light from the oncoming vehicle is known andthe locations of the driver's eyes are known, it is a matter of simpletrigonometry to determine which areas of the windshield matrix should bedarkened to improve a filter between the headlights and the driver'seyes. This is accomplished by processor 101. A separate control system,not shown, located on the instrument panel, or at some other convenientlocation, allows the driver to select the amount of darkeningaccomplished by the system from no darkening to maximum darkening. Inthis manner, the driver can select the amount of light that is filteredto suit his particular physiology. The sensor 410 can either be designedto respond to a single light source or to multiple light sources to besensed and thus multiple portions of the vehicle windshield to bedarkened.

As mentioned above, the calculations of the location of the driver'seyes using acoustic systems may be in error and therefore provision mustbe made to correct for this error. One such system permits the driver toadjust the center of the darkened portion of the windshield to correctfor such errors through a knob on the instrument panel, steering wheelor other convenient location. Another solution permits the driver tomake the adjustment by slightly moving his head. Once a calculation asto the location of the driver's eyes has been made, that calculation isnot changed even though the driver moves his head slightly. It isassumed that the driver will only move his head to center the darkenedportion of the windshield to optimally filter the light from theoncoming vehicle. The monitoring system will detect this initial headmotion and make the correction automatically for future calculations.

Electro-chromic glass is currently used in rear view mirrors to darkenthe entire mirror in response to the amount of light striking anassociated sensor. This substantially reduces the ability of the driverto see objects coming from behind his vehicle. If one rear-approachingvehicle, for example, has failed to dim his lights, the mirror will bedarkened to respond to the light from that vehicle making it difficultfor the driver to see other vehicles that are also approaching from therear. If the rear view mirror is selectively darkened on only thoseportions which cover the lights from the offending vehicle, the driveris able to see all of the light coming from the rear whether the sourceis bright or dim. This permits the driver to see all of the approachingvehicles not just the one with bright lights.

Such a system is illustrated in FIG. 12 where rear view mirror 460 isequipped with electro-chromic glass, or comprises a liquid crystal film,having the capability of being selectively darkened, e.g., at area 419.Associated with mirror 460 is a light sensor 462 that determines thedirection of light 412 from the headlights of rear approaching vehicle405. In the same manner as above, transducers 231, 232 and 233 determinethe location of the eyes of the driver 210. The signals from both sensorsystems, 231, 232 plus 233 and 462, are combined in processor 101, wherea determination is made as to what portions of the mirror should bedarkened, e.g., area 419. Appropriate currents are then sent to themirror in a manner similar to the windshield system described above.

Seatbelts are most effective when the upper attachment point to thevehicle is positioned vertically close to the shoulder of the occupantbeing restrained. If the attachment point is too low the occupantexperiences discomfort from the rubbing of the belt on his shoulder. Ifit is too high, the occupant may experience discomfort due to therubbing of the belt against his neck and the occupant will move forwardby a greater amount during a crash which may result in his head strikingthe steering wheel. Women in particular experience discomfort from animproperly adjusted seatbelt anchorage point. For these reasons, it isdesirable to have the upper seatbelt attachment point located slightlyabove the occupant's shoulder. To accomplish this for various sizedoccupants, the location of the occupant's should must be known which canbe accomplished by the vehicle interior monitoring system describedherein. Such a system is illustrated in FIG. 13 that is a side view of aseatbelt anchorage adjustment system. In this system, a transmitter andreceiver (transducer) 520 is positioned in a convenient location, suchas the headliner, located above and to the outside of the occupant'sshoulder. A narrow elliptical beam 521 of energy is transmitted fromtransducer 520 in a manner such that it illuminates the occupant'sshoulder and headrest. An appropriate pattern recognition system asdescribed above is then used to determine the location and position ofthe shoulder. This information is fed to the seatbelt anchorage heightadjustment system 528, shown schematically, which moves the attachmentpoint 529 to the optimum vertical location.

Acoustic resonators are devices that resonate at a preset frequency whenexcited at that frequency. If such a device, which has been tuned to 40kHz, is subjected to ultrasonic radiation at 40 kHz, for example, itwill return a signal that is much stronger than the reflected radiation.If such a device is placed at a particular point in the passengercompartment of a vehicle, the returned signal can be easily identifiedas a high magnitude narrow signal at a particular point in time which isproportional to the distance from the resonator to the receiver. Sincethis device can be easily identified, it provides a particularlyeffective method of determining the distance to a particular point inthe vehicle passenger compartment. If several such resonators are usedthey can be tuned to slightly different frequencies and thereforeseparated and identified by the circuitry. Using such resonators thepositions of various objects in the vehicle can be determined. In FIG.14 for example, three such resonators are placed on the vehicle seat andused to determine the location of the front and back of the seat and thetop of the seat back. In this case, transducers 231 and 232, mounted inthe A-pillar 662, are used in conjunction with resonators 641, 642 and643 to determine the position of the seat. This information is then fedto the seat memory and adjustment system, not shown, eliminating thecurrently used sensors that are placed typically beneath the seatadjacent the seat adjustment motors. In the conventional system, theseat sensors must be wired into the seat adjustment system and are proneto being damaged. By using the vehicle interior monitoring system alonewith inexpensive passive resonators, the conventional seat sensor can beeliminated resulting in a cost saving to the vehicle manufacture. Anefficient reflector, such as a parabolic shaped reflector, can be usedin a similar manner as the resonator.

Resonators or reflectors, of the type described above can be used formaking a variety of position measurements in the vehicle. Theseresonators are made to resonate at a particular frequency. If the numberof resonators increases beyond a reasonable number, dual frequencyresonators can be used. A pair of frequencies is then used to identify aparticular location. Alternately, resonators tuned to a particularfrequency can be used in combination with special transmitters, whichtransmit at the tuned frequency, which are designed to work with aparticular resonator or group of resonators. The cost of the transducersis sufficiently low to permit special transducers to be used for specialpurposes. The use of resonators which resonate at different frequenciesrequires that they be illuminated by radiation containing thosefrequencies.

Another application for a resonator of the type described is todetermine the location of the seatbelt and therefore determine whetherit is in use. If it is known that the occupants are wearing seatbelts,the airbag deployment threshold can be increased since the airbag is notneeded in low velocity accidents if the occupants are already restrainedby seatbelts. This will reduce the number of deployments for cases wherethe airbag provides little or no improvement in safety over theseatbelt. FIG. 15, for example, shows the placement of a resonator 602onto the front surface of the seatbelt where it can be sensed by thetransducers 231 and 232. Such a system can also be used to positivelyidentify the presence of a rear facing child seat in the vehicle. Inthis case the resonator 603 is placed onto the forward most portion ofthe child seat, or in some other convenient position, as shown in FIG.1A.

Other uses for such resonators include placing them on doors and windowsin order to determine whether either is open or closed. In FIG. 16A, forexample, such a resonator 604 is placed onto the top of the window andis sensed by transducers 611 and 612. In this case, transducers 611 and612 also monitor the space between the edge of the window glass and thetop of the window opening. Many vehicles now have systems which permitthe rapid opening of the window, called "express open", by a momentarypush of a button. For example, when a vehicle approaches a tollbooth,the driver needs only touch the window control button and the windowopens rapidly. Some automobile manufacturers do not wish to use suchsystems for closing the window, called "express close", because of thefear that the hand of the driver, or of a child leaning forward from therear seat, or some other object, could get caught between the window andwindow frame. If the space between the edge of the window and the windowframe were monitored with an interior monitoring system, this problemcan be solved. The presence of the resonator or reflector 604 on the topof the window glass also gives a positive indication of where the topsurface is and reflections from below that point can be ignored.

Various design variations of the window monitoring system are possibleand the particular choice will depend on the requirements of the vehiclemanufacturer and the characteristics of the vehicle. Two systems will bebriefly described here.

In the first example shown in FIG. 16, a single transmitter/receiver(transducer) 613 is used in place in and located centrally midwaybetween the transducers 611 and 612 shown in FIG. 16A. A recording ofthe output of transducer 613 is made of the open window without anobject in the space between the window edge and the top of the windowframe. When in operation, the transducer 613 receives the return signalfrom the space it is monitoring and compares that signal with the storedsignal referenced above. This is done by processor 601. If thedifference between the test signal and the stored signal indicates thatthere is a reflective object in the monitored space, the window isprevented from closing in the express close mode. If the window is partway up, a reflection will be received from the edge of the window glasswhich, in most cases, is easily identifiable from the reflection of ahand for example. A simple algorithm based on the intensity of thereflection in most cases is sufficient to determine that an objectrather than the window edge is in the monitored space. In other cases,the algorithm is used to identify the window edge and ignore thatreflection and all other reflections which are lower (i.e. later intime) than the window edge. In all cases, the system will default in notpermitting the express close if there is any doubt. The operator canstill close the window by holding the switch in the window closingposition and the window will then close slowly as it now does invehicles without the express close feature.

In the second system, two transducers 611 and 612 are used as shown inFIG. 16A and the processor 601 comprises a neural network. In thisexample the system is trained for all cases where the window is down andat intermediate locations. In operation, the transducers monitor thewindow space and feed the received signals to processor 601. As long asthe signals are similar to one of the signals for which the network wastrained, the express close system is enabled. As before, the default isto suppress the express close.

The use of a resonator, or reflector, to determine whether the vehicledoor is properly shut is illustrated in FIG. 17. In this case, theresonator 702 is placed in the B-pillar in such a manner that it isshielded by the door, or by a cover or other inhibiting mechanism (notshown) engaged by the door, and prevented from resonating when the dooris closed. Resonator 702 provides waves 704. If transducers such as 231and 232 in FIG. 3 are used in this system, the closed-door conditionwould be determined by the absence of a return signal from the B-pillar702 resonator. This system permits the substitution of an inexpensiveresonator for a more expensive and less reliable electrical switch.

The use of an acoustic resonator has been described above. For thosecases where an infrared laser system is used, an optical mirror wouldreplace the mechanical resonator used with the acoustic system. In theacoustic system, the resonator can be any of a variety of tunedresonating systems including an acoustic cavity or a vibratingmechanical element.

A neural network, or other pattern recognition system, can be trained torecognize certain people as permitted operators of a vehicle. In thiscase, if a non-recognized person attempts to operate the vehicle, thesystem can disable the vehicle and/or sound an alarm as illustrated inFIG. 18. In this figure the sensing transducers are shown as before as231A, 232A and 233A, the alarm system schematically as 708 and the alarmas 705. Since it is unlikely that an unauthorized operator will resemblethe authorized operator, the neural network system can be quite tolerantof differences in appearance of the operator. The system defaults towhere a key must be used in the case that the system doesn't recognizethe driver or the owner wishes to allow another person to operate thevehicle. The transducers 231A, 232A and 233A are sensitive to infraredradiation and the operator is illuminated with infrared waves fromtransducer 231A. This is necessary due to the small size of the featureswhich need to be recognized for high accuracy of recognition. Analternate system uses an infrared laser, which can be 231A in FIG. 18,to illuminate the operator and a CCD or CMOS device, which can berepresented as 232A in FIG. 18, to receive the reflected image. In thiscase the recognition of the operator is accomplished using a patternrecognition system such as described in Popesco, V. and Vincent, J. M."Location of Facial Features Using a Boltzmann Machine to ImplementGeometric Constraints", Chapter 14 of Lisbon, P. J. G. and Taylor, M. J.Editors, Techniques and Applications of Neural Networks, Ellis HorwoodPublishers, New York, 1993. In the present case a larger CCD elementarray containing 100,000 or more elements would in many cases be usedinstead of the 16 by 16 or 256 element CCD array used by Popesco andVincent.

Once a vehicle interior monitoring system employing a sophisticatedpattern recognition system, such as a neural network or fuzzy logicsystem, is in place, it is possible to monitor the motions of the driverover time and determine if he is falling asleep or has otherwise becomeincapacitated. In such an event, the vehicle can be caused to respond ina number of different ways. One such system is illustrated in FIG. 19and consists of a monitoring system having transducers 231, 232 and 233plus microprocessor 101, such as shown in FIG. 7A, programmed to comparethe motions of the driver over time and trained to recognize changes inbehavior representative of becoming incapacitated. If the systemdetermines that there is a reasonable probability that the driver hasfallen asleep, for example, then it can turn on a warning light shownhere as 805 or send a warning sound. If the driver fails to respond tothe warning by pushing a button 806, for example, then the horn andlights can be operated in a manner to warn other vehicles and thevehicle brought to a stop. One novel approach, not shown, would be touse the horn as the button 806. For a momentary depression of the horn,for this case, the horn would not sound. Naturally other responses canalso be programmed.

An even more sophisticated system of monitoring the behavior of thedriver is to track his eye motions using such techniques as aredescribed in: Freidman et al., U.S. Pat. No. 4,648,052 "Eye TrackerCommunication System"; Heyner et al, U.S. Pat. No. 4,720,189 "EyePosition Sensor"; Hutchinson U.S. Pat. No. 4,836,670 "Eye MovementDetector"; and , Hutchinson U.S. Pat. No. 4,950,069 "Eye MovementDetector With Improved Calibration and Speed", all of which are includedherein by reference. The detection of the impaired driver in particularcan be best determined by these techniques. Also, in a similar manner asdescribed in these patents, the motion of the driver's eyes can be usedto control various systems in the vehicle permitting hands off controlof the entertainment system, heating and air conditioning system or allof the other systems described above. Although some of these systemshave been described in the afore mentioned patents, none have made useof neural networks for interpreting the eye movements.

In most of the applications described above, a single frequency energywas used to illuminate various occupying items of the passengercompartment. This was for illustrative purposes only and this inventionis not limited to single frequency illumination. In many applications,it is useful to use several discrete frequencies or a band offrequencies. In this manner considerably greater information is receivedfrom the reflected illumination permitting greater discriminationbetween different classes of objects. In general each object will havedifferent reflectivities at each frequency. Also, the differentresonators placed at different positions in the passenger compartmentcan now be tuned to different frequencies making it easier to isolateone resonator from another.

Although several preferred embodiments are illustrated and describedabove, there are possible combinations using other geometries, sensors,materials and different dimensions for the components that perform thesame functions. This invention is not limited to the above embodimentsand should be determined by the following claims.

We claim:
 1. In a motor vehicle having an interior passenger compartment receivable of an occupant, a windshield and a rear view mirror, a system for optimizing the comfort of the occupant comprising:first determining means for determining the direction of a bothersome stimulus; second determining means for determining the location of the sensory organ of the occupant receptive to the stimulus and, activation means coupled to said first and second determining means responsive to the determined direction of the stimulus and the determined location of the sensory organ of the occupant receptive to the stimulus for automatically activating a countereffect to the stimulus in order to diminish the effect of the stimulus on the sensory organ of the occupant.
 2. The invention in accordance with claim 1, wherein the stimulus is an external light source such that said first determining means are structured and arranged to determine the direction of the external light source and said second determining means are structured and arranged to determine the location of the eyes of the occupant.
 3. The invention in accordance with claim 2, wherein said activation means are structured and arranged to activate a light filter between the external light source and the eyes of the occupant.
 4. The invention in accordance with claim 3, wherein said filter comprises electro-chromic glass arranged in connection with windows of the vehicle.
 5. The invention in accordance with claim 3, wherein said filter comprises a liquid crystal film arranged in connection with windows of the vehicle.
 6. The invention in accordance with claim 3, wherein the windshield comprises said filter.
 7. The invention in accordance with claim 6, wherein said filter is structured and arranged to limit the amount of light passing through only a selected portion of the windshield.
 8. The invention in accordance with claim 3, wherein the rear view mirror comprises said filter.
 9. The invention in accordance with claim 8, wherein said filter is structured and arranged to limit the amount of light reflected from only a selected portion of the rear view mirror.
 10. The invention in accordance with claim 1, wherein the stimulus is a source of unwanted sound such that said first determining means are structured and arranged to determine the presence and direction of the source of unwanted sound and said second determining means are structured and arranged to determine the location of the ears of the occupant.
 11. The invention in accordance with claim 10, wherein said activation means are structured and arranged to generate and transmit sound waves to cancel the unwanted sound at the ears of the occupant.
 12. The invention in accordance with claim 11, further comprisingthird determining means for determining the direction of an external light source, fourth determining means for determining the location of the eyes of the occupant, and additional activation means coupled to said third and fourth determining means responsive to the determined direction of the external light source and the determined location of the eyes of the occupant for automatically activating a light filter between the external light source and the eyes of the occupant.
 13. The invention in accordance with claim 1, wherein said second determining means comprises a transmitter for transmitting waves toward the occupant, a plurality of receivers for receiving waves reflected from the occupant and a processor for processing the received reflected waves into an indication of the location of the sensory organ of the occupant based in part on pattern recognition techniques.
 14. In a motor vehicle having an interior passenger compartment receivable of an occupant, a method for optimizing the comfort of the occupant comprising the steps of:determining the direction of a bothersome stimulus; determining the location of the sensory organ of the occupant receptive to the stimulus, and in consideration of the determined direction of the stimulus and the determined location of the sensory organ of the occupant receptive to the stimulus, automatically activating a countereffect to the stimulus in order to diminish the effect of the stimulus on the sensory organ of the occupant.
 15. The method in accordance with claim 14, wherein the stimulus is an external light source such that the location of the eyes of the occupant are determined, said step of activating a countereffect to the stimulus comprising the step of activating a light filter between the external light source and the eyes of the occupant.
 16. The method in accordance with claim 15, wherein said filter comprises electro-chromic glass arranged in connection with windows of the vehicle.
 17. The method in accordance with claim 15, wherein said filter comprises a liquid crystal film arranged in connection with windows of the vehicle.
 18. The method in accordance with claim 15, further comprising the step of arranging said filter in association with a windshield of the vehicle.
 19. The method in accordance with claim 18, wherein said filter is structured and arranged to limit the amount of light passing through only a selected position of the windshield.
 20. The method in accordance with claim 15, further comprising the step of arranging said filter in association with a rear view mirror of the vehicle.
 21. The method in accordance with claim 15, wherein said filter is structured and arranged to limit the amount of light reflected from only a selected portion of the rear view mirror.
 22. The method in accordance with claim 14, wherein the stimulus is a source of unwanted sound such that the location of the ears of the occupant are determined, said step of activating a countereffect to the stimulus comprising the step of generating and transmitting sound waves to cancel the unwanted sound at the ears of the occupant.
 23. The method in accordance with claim 22, further comprising the steps of:determining the direction of an external light source, determining the location of the eyes of the occupant, and in consideration of the determined direction of the external light source and the determined location of the eyes of the occupant, automatically activating a light filter between the external light source and the eyes of the occupant.
 24. The method in accordance with claim 14, wherein the step of determining the location of the sensory organ comprises the steps of:transmitting waves toward the occupant, receiving waves reflected from the occupant at a plurality of receivers, and processing the received reflected waves into an indication of the location of the sensory organ of the occupant based in part on pattern recognition techniques.
 25. In a motor vehicle having an interior passenger compartment receivable of an occupant, a windshield and a rear view mirror, a system for optimizing the comfort of the occupant comprising:first determining means for determining the direction of an external light source, second determining means for determining the location of the eyes of the occupant, first activation means coupled to said first and second determining means responsive to the determined direction of the external light source and the determined location of the eyes of the occupant for automatically activating a light filter between the external light source and the eyes of the occupant, third determining means for determining the presence and direction of a source of unwanted sound, fourth determining means for determining the location of the ears of the occupant, second activation means coupled to said third and fourth determining means responsive to the determined direction of the unwanted sound and the determined location of the ears of the occupant for automatically generating and transmitting sound waves to cancel the unwanted sound at the ears of the occupant. 